Connect-DbaInstance - Give the cloned server ownership of its database connection - #10584
Connect-DbaInstance - Give the cloned server ownership of its database connection#10584andreasjordan wants to merge 4 commits into
Conversation
|
seems straightforward but there must have been a reason we did it so weirdly. at the same time, it does cause issues. im going to ask gpt to investigate why we likely did it and what the consequences are of the change. |
|
GPT Pro said on my behalf: I dug through the history because this code was weird enough that I didn't believe we did it accidentally. The weird code had real reasonsThe original
Then more fixes accumulated around it:
So no, we didn't do this for funsies. But this PR found a real bugThe ownership problem described in #10584 checks out. Basically: A → Copy to B → GetDatabaseConnection creates C → return C But SMO can open/use B while creating C. We then throw away our reference to B and return a Server containing C.
So B can sit there for the life of the process holding a database lock. That explains both the accumulating sessions and the bizarre later failures trying to get an exclusive lock on For normal dbatools connections, changing this to: is much cleaner. It also makes several old workarounds unnecessary: no premature connection, no lost One thing I think this PR currently breaksWe also support creating a Server from a raw Those SMO contexts have an explicit connection string. So please add a regression test for: I expect the second call to fail on the new CI doesn't catch this because we test SqlConnection → Server and separately Server → different Database, but not SqlConnection → Server → different Database. What I'd change before mergingKeep the new For explicit-connection-string contexts, construct the final database-specific context directly with the target database already in I'd also add coverage for Finally, I'd make the leak regression run several cycles. With pooling, a sleeping SQL session remaining is legitimate. What matters is that the session count stabilizes instead of growing every time, which your eight-cycle test already demonstrates really well. Bottom lineThe PR is fixing a real bug and I prefer the new design. I wouldn't revert to the old approach. I just want the raw |
641a950 to
bf69183
Compare
|
Thank you for putting this through GPT Pro — the The report was right, and it was a regression
The explanation of why CI misses it is right too: we cover Two more of the same, and they are not new
What it doesAs advised, it does not fall back to There is no way to detect the two kinds of context up front — every property reads fine on both, and Two things that cost me a while and are worth knowing:
On preserving authentication: the password survives. TestsA new Verified to have teeth by reverting the source and keeping the tests: each fails with exactly the exception it guards against. The leak regression now runs five cycles, as suggested. With pooling a single sleeping session that stays behind is legitimate, and only growth proves an orphaned connection, so one cycle could not tell the two apart. Against
One note on the diff: the DAC block was restructured, so a few pre-existing single-quoted strings in it became double-quoted. That is the repo style rule and the pre-commit style hook refuses to touch those lines otherwise. This text was created by Claude and reviewed by Andreas Jordan. |
|
giving this a bit more review bc connect-dbainstance is so important -- needs more integration tests VerdictRequest changes. The leak diagnosis is correct, and replacing Findings1. High, blocker until disproved: an already-open SQL-authenticated
|
…e connection When an existing server object is passed in together with a different -Database, the connection context is copied and the database connection was then created with GetDatabaseConnection. That opens the connection on the intermediate copy and returns a different ConnectionContext, so the server object we hand back never owns the connection. Disconnect-DbaInstance can only reach the context of the server it is given, so nothing ever closed it. The session therefore stayed open for the life of the process, sitting in the target database and holding a shared lock on it. On model that is enough to make a later CREATE DATABASE on the same instance fail with "Could not obtain exclusive lock on database model", which showed up as an intermittent failure in whatever test file happened to run next. Setting DatabaseName on the copy keeps the connection with the context that the returned server owns, so Disconnect-DbaInstance closes it. This is also what the connection string paths of this command already do, and it does not reset StatementTimeout, so the save and restore around the old call is no longer needed. Measured against one instance, connecting to a database and disconnecting again: before, four sessions were opened and one closed, leaving three behind including one parked in the database. Now two are opened and the database one is closed again. (do Connect-DbaInstance) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…nto that connection string A ServerConnection built from a SqlConnection has its connection string set explicitly, and SMO then refuses to let DatabaseName, NonPooledConnection and ServerInstance be assigned. The previous commit started to rely on the DatabaseName assignment, which broke SqlConnection -> Server -> -Database. The other two were already broken on development: -Database -NonPooledConnection and -DedicatedAdminConnection both threw for such a server. All three now fall back to the connection string, where they are Initial Catalog, Pooling and Data Source. Falling back to GetDatabaseConnection was not an option because that is the leak this branch is about. Initial Catalog is part of the pool key, so the reason #9505 forced a non pooled connection still holds. The copy is disconnected before its connection string is set, because setting it on an open connection does not move it. The copy is a session of its own, so the caller is not affected. Tests: a Context for cloning from a server that was created from a SqlConnection, which is the shape CI never covered - it tests SqlConnection to Server and Server to another Database, but never chained. The leak regression now runs five cycles, because with pooling a single sleeping session is legitimate and only growth proves an orphaned connection. Against development it fails with "Expected 4, but got 7". (do Connect-DbaInstance)
…f rebuilding its string Answers the review on #10584. All three findings were reproduced in the lab. The password loss is real and it was a regression. Once a SqlConnection has been opened with Persist Security Info=False, SqlClient hides the password, so the connection string that can be read back no longer carries one. Rebuilding from it produced a clone that failed on first use with "Login failed for user". Verified against a real SQL Server login. So -Database no longer touches the connection string at all. The copy already holds a working connection of its own - a different SPID from the server that was passed in - and the database is switched on it with ChangeDatabase. Nothing has to be rebuilt, so nothing that lives on the SqlConnection rather than in its string can be lost, and the copy stays the context the returned server owns, which is what this branch is about. The two settings that genuinely need a new connection, -NonPooledConnection and -DedicatedAdminConnection, still rebuild the string, and now refuse rather than hand back a server that cannot log in when the password is no longer readable. The caller is told to use the instance name with -SqlCredential instead. TrustServerCertificate does not reach a fixed connection string either. Assigning the property succeeds and reads back as True while the string still says False, so the localhost DAC would have lost the trust that #10254 added. It is put into the string now, and ApplicationIntent with it, which had the same silent mismatch. Tests: a Context for an already open SQL authenticated SqlConnection, which asserts that the password really is hidden, that the clone works and runs as that login, and that the case needing a new connection is refused with a readable message. Both fail against the previous revision with "Login failed for user". The leak regression now requires every cycle to report the same number of sessions rather than only the first and the last, so a sequence like 4, 5, 4, 5, 4 fails. Not covered: the localhost DAC path, because every instance in the lab used for this is remote. The keyword is added from the same branch that sets the property, and the non-local DAC path is tested. (do Connect-DbaInstance)
bf69183 to
00ed917
Compare
|
All three findings reproduced in the lab, and the first one was right to be called a blocker — it is a regression this branch introduced. Pushed a revision that changes the approach rather than patching the fallback. 1. The password loss is realReproduced against a real SQL Server login, with an already open Exactly the chain described. The fix is to stop rebuilding the string for if ($connContext.SqlConnectionObject.State -ne "Open") {
$connContext.Connect()
}
$connContext.SqlConnectionObject.ChangeDatabase($Database)Nothing is rebuilt, so nothing that lives on the
The two settings that genuinely need a new connection — The regression test you asked for is in, shaped as suggested. Against the previous revision both of its cases fail with 2. The DAC trust setting is confirmed, and it is worse than a DAC problemConfirmed, and it is not specific to The property assignment succeeds and reads back, while the string is untouched. One honest gap: the localhost DAC path is not covered by a test, because every instance in the lab used for this work is remote, so 3. Session-count instabilityFair, ($countPerCycle | Select-Object -Unique).Count | Should -Be 1Measured over six cycles on both paths, from an instance name and from a Where it stands
Also rebased onto current This text was created by Claude and reviewed by Andreas Jordan. |
…ch CI can reach The CI instances are on the machine running the tests, so the local DAC path can be exercised there even though it cannot be in a lab of remote instances. The test skips where the instance is not local and runs on the runners, which is where the coverage was missing. It starts from Trust Server Certificate=False on purpose. With True it would pass even if the command did nothing, which is what made the earlier DAC test blind to this: on a context whose connection string is fixed, assigning TrustServerCertificate succeeds and reads back as True while the string still says False, and the string is what the new connection is built from. The assertion reads the setting back through a connection string builder rather than matching the string. A builder keeps the spelling it was given, so the same setting comes out as "Trust Server Certificate=True" or "TrustServerCertificate=True" depending on how the caller wrote it, and matching the second one against a string built from the first fails for no good reason. (do Connect-DbaInstance)
|
The gap I flagged in the previous comment is closed: the CI instances are on the machine running the tests, so the local DAC path can be exercised there even though a lab of remote instances cannot reach it. Added a test that skips where the instance is not local and runs on the runners. It starts from $localDacConnectionString = "Data Source=$($TestConfig.InstanceMulti1);Integrated Security=True;Encrypt=False;Trust Server Certificate=False"With One thing worth knowing for anyone asserting on connection strings, which caught me while writing it: a So the obvious assertion would have failed on CI for a reason that has nothing to do with the fix. The test reads the setting back through a builder instead, which normalises it: $cloneStringBuilder = New-Object -TypeName Microsoft.Data.SqlClient.SqlConnectionStringBuilder -ArgumentList $serverClone.ConnectionContext.ConnectionString
$cloneStringBuilder["Trust Server Certificate"] | Should -BeTrue
$cloneStringBuilder["Data Source"] | Should -Match "^ADMIN:localhost"
This text was created by Claude and reviewed by Andreas Jordan. |
|
CI is green, and the local DAC gap from the previous comment is now actually closed rather than just intended. The MULTI lane failure was infrastructure, not a test. The job ran exactly 10 minutes, steps 4 to 7 finished with no conclusion at all, and its log blob was never uploaded ( The local DAC test did run there. So the Full result for the branch: 22 checks, 21 pass, 1 skipping (the One aside that came out of reading those logs and is worth recording somewhere less transient: the ci-azure MULTI lane is This text was created by Claude and reviewed by Andreas Jordan. |
When an existing server object is passed in together with a different
-Database, the connection context is copied and the database connection was then created withGetDatabaseConnection:GetDatabaseConnectionopens the connection on the intermediate copy and returns a differentConnectionContext, so the server object that is handed back never owns that connection.Disconnect-DbaInstancecan only reach the context of the server it is given, so nothing ever closed it.The session therefore stayed open for the life of the process, sitting in the target database and holding a shared
DATABASElock on it. Onmodelthat is enough to make a laterCREATE DATABASEon the same instance fail:which surfaced as an intermittent failure in whatever test file happened to run next, with no connection to the command that caused it.
Where the connection goes
Running the steps of the old code one at a time and counting sessions on the instance after each:
Step 3 opens the connection on the copy. Step 3 also reassigns
$connContext, which drops the only reference to the object that owns it. Step 6 shows the consequence: the server we return has nothing to close. Step 7 shows who did own it - and that disconnecting the copy is not a usable fix either, because that is the working connection.The change
Setting
DatabaseNameon the copy keeps the connection with the context that the returned server owns, soDisconnect-DbaInstancecloses it. This is also what the connection string paths of this command already do, so the server object path now behaves like the others.History of the line, and why each earlier fix still holds
The line has been touched four times, and none of the reasons are lost by this change:
ConnectionContext.Copy().GetDatabaseConnection($Database), next to a# TODO: Do we have to check if its the same database?GetDatabaseConnectionopens the connection and later property assignments would come too lateDatabaseNamedoes not open a connection - the trace above shows nothing opens until the first query. The assignment is kept in the same last position anywayGetDatabaseConnectionresetsStatementTimeoutStatementTimeoutis set earlier on the copy and simply stays.clones when using parameter StatementTimeoutcovers itGetDatabaseConnection($Database, $false), to force a non-pooled connection. Without it,Backup-DbaDatabasegot a cached connection to an already dropped database and the context change silently did not happen. Also added the warning whenCurrentDatabasedoes not match#9505 is the one worth care, because
$falseforced a non-pooled connection and settingDatabaseNamedoes not. It holds becauseDatabaseNameputsInitial Cataloginto the connection string, so the database is part of the pool key: a pooled connection then comes from that database's pool instead of being taken from the original database's pool and switched.Replaying the exact #9505 scenario against this change - create a database, connect with it as context, drop it, then ask for a clone on
master:And with pooling left on everywhere, which is the case
$falseused to opt out of:The test #9505 added,
clones when using Backup-DabInstace, is still in the suite and passes.Measured
The clearest way to see it is to repeat the call. Eight cycles of connect to a database from an existing server object, each followed by
Disconnect-DbaInstance, counting the module's sessions on the instance after every cycle:Before - sessions accumulate without bound, and every new one sits in the target database:
After - flat, and the same spids are reused every cycle:
That is the difference between an orphaned non-pooled connection, which nothing can ever reuse or close, and a pooled one that goes back to the pool on disconnect. The sessions that remain after the fix are the pool's: they stay
sleeping, they are reused by the next call, and their number does not grow. Connecting from a plain instance name behaves the same way in both versions, which is why that path never had the problem.At suite scale the same difference: the full 744 file run of 2026-08-15 had 69 files failing with
Timeout expired ... prior to obtaining a connection from the pool, and the same run with this fix had zero.Commands that read a per database view through
Invoke-DbaQuery -Databasestop leaking without being touched.Get-DbaDbQueryStoreOption -Database modelwent from three leaked sessions, one of them inmodel, to one leaked session and none inmodel.Tests
tests/Connect-DbaInstance.Tests.ps1gets a regression test that connects to a database, disconnects again and asserts the session count is back where it started. It was verified to fail against the old implementation (Expected 4, but got 5) and to be the only test that fails there.Run against SQL Server 2025, including the commands that use this path:
Connect-DbaInstanceInvoke-DbaQueryRemove-DbaDbDataGet-DbaUserPermissionGet-DbaDbRecoveryModelGet-DbaDbQueryStoreOptionDisconnect-DbaInstance,Remove-DbaDbAsymmetricKey,Remove-DbaDbCertificate,Remove-DbaDbEncryptionKey106 tests, no failures.
This text was created by Claude and reviewed by Andreas Jordan.
🤖 Generated with Claude Code